Skip to content

Rawaudio dev#3653

Open
dingodoppelt wants to merge 58 commits intojamulussoftware:mainfrom
dingodoppelt:rawaudio-dev
Open

Rawaudio dev#3653
dingodoppelt wants to merge 58 commits intojamulussoftware:mainfrom
dingodoppelt:rawaudio-dev

Conversation

@dingodoppelt
Copy link
Copy Markdown
Contributor

@dingodoppelt dingodoppelt commented Apr 17, 2026

Add a new "raw" audio quality setting

This PR adds uncompressed audio ("raw") to the quality settings so there is no Opus compression along the way
Discussion in #3654

This feature improves latency as well. I gained 2ms by using uncompressed audio while having a better audio quality.

CHANGELOG: Add uncompressed audio transmission - dedicated to the memory of Hans Petter Selasky (1982 - 2023)

Does this change need documentation? What needs to be documented and how?
Corresponding PR in jamulussoftware/jamuluswebsite #1133

Checklist

  • I've verified that this Pull Request follows the general code principles
  • I tested my code and it does what I want
  • My code follows the style guide
  • I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
  • I've filled all the content above

@dingodoppelt dingodoppelt marked this pull request as ready for review April 19, 2026 06:54
@ann0see ann0see added this to the Release 4.0.0 milestone Apr 20, 2026
@ann0see ann0see added this to Tracking Apr 20, 2026
@github-project-automation github-project-automation Bot moved this to Triage in Tracking Apr 20, 2026
Comment thread src/clientsettingsdlg.cpp Outdated
Comment thread src/util.h
Comment thread src/client.cpp
@ann0see
Copy link
Copy Markdown
Member

ann0see commented Apr 20, 2026

I'd prefer not to check for the Jamulus version number but rather based on capabilities - we don't have 4.0.0 out yet and it might break during the dev process.

@dingodoppelt
Copy link
Copy Markdown
Contributor Author

I'd prefer not to check for the Jamulus version number but rather based on capabilities - we don't have 4.0.0 out yet and it might break during the dev process.

I wanted to reuse information already available as much as possible so I just added the code where there were version checks already implemented. (For sequence number and pan feature)
Capabilities would be nice but also would require more changes to client, channel, server and protocol which I don't really have an idea on how to make that backwards compatible. We should rather replace all version checks with some capabilities struct that client and server can agree upon so everything lands in one place. I just don't feel like the right person to take on that challenge and rather pursue my hacky approach, as long as it works for everybody.
The version check with 4.0.0 could be replaced by a point release 3.11.1 and would work right away.

@ann0see
Copy link
Copy Markdown
Member

ann0see commented Apr 20, 2026

Tested it and yes, the noise would be unacceptable. What is our fallback if max is selected but the server doesn't support it?

@dingodoppelt
Copy link
Copy Markdown
Contributor Author

dingodoppelt commented Apr 20, 2026

Tested it and yes, the noise would be unacceptable. What is our fallback if max is selected but the server doesn't support it?

I just noticed that if you connect to a server with Max selected you get the noise unless you switch audio quality again while connected. The server code is fine and doesn't need changes, I misplaced the check for my introduced bRawAudioSupported in the client code. I'll have a closer look
Edit: Funny, the noise doesn't happen on legacy servers, only on rawaudio :D

Copy link
Copy Markdown
Contributor Author

@dingodoppelt dingodoppelt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is in a working state now and the only issue remaining is the infrequent unclean exits especially on macOS. As it might even be unrelated to this pull request and I don't have an idea what causes it, I'll set this PR to "ready for review" now.

Comment thread src/client.cpp Outdated
Comment thread src/client.cpp
Comment thread src/server.cpp Outdated
}

const int iOffset = iB * SYSTEM_FRAME_SIZE_SAMPLES * vecNumAudioChannels[iChanCnt];
// Recognise a raw audio packet by its size
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understood these sentinel bytes come from the opus codec itself and are not deliberately set by Jamulus as a message ID of sorts. I'd have to overwrite actual audio bytes for that to work with my code. Or am I wrong here?

Comment thread src/client.cpp
Comment thread src/client.cpp
Comment thread src/server.cpp Outdated
Comment thread src/server.cpp
Comment thread src/util.h
Comment thread src/client.cpp
Comment thread src/client.cpp
@dingodoppelt dingodoppelt marked this pull request as ready for review April 26, 2026 13:14
Comment thread src/server.cpp
@rdica
Copy link
Copy Markdown
Contributor

rdica commented Apr 29, 2026

@dingodoppelt I've received reports of users on Mac and Windows that have small buffers enabled in their rawaudio clients being unable to hear anything or hear strange audio on rawaudio enabled servers that do not have small buffers enabled. They report when they disable that feature in the client they can hear audio fine. I don't have Mac or Windows to test, and I don't experience the issue in Linux.
From one Windows user I was able to get the client version as 3.11.1dev-4b8bf6a.
Server version is 3.11.1dev-09ae6dfc:1777232839 on Linux.

@softins
Copy link
Copy Markdown
Member

softins commented Apr 29, 2026

I've received reports of users on Mac and Windows that have small buffers enabled in their rawaudio clients being unable to hear anything or hear strange audio on rawaudio enabled servers that do not have small buffers enabled. They report when they disable that feature in the client they can hear audio fine.

I've just checked this out locally, with a server on Raspberry Pi and a client on Mac, both built from the latest rawaudio-dev at 3f059b3

If I start the server with -F, then all buffer sizes in the client, with small network buffers either off or on, work fine.

If I start the server without -F, then 2.67ms (64) with Small Network Buffers enabled in the client results in very distorted audio. All other settings still work fine.

However, I have also just checked with the Opus quality settings: High, Normal and Low. In those cases, setting 2.67ms (64) with Small Buffers enabled, connected to a server running without -F, also gives the same strange audio. I haven't yet looked at what -F actually does in the code, but it suggests that this issue is not specifically related to Raw Audio.

From previous tests I've done in standard Jamulus, the "Small Network Buffers" checkbox does nothing when set to 5.33ms (128) or 10.67ms (256). If the checkbox is off, then 2.67ms (64) actually sends exactly the same as 5.33ms (128). It's only if the checkbox is on, that 2.67ms (64) sends smaller packets.

@ann0see
Copy link
Copy Markdown
Member

ann0see commented Apr 29, 2026

Yes. Can confirm the distorted audio without -F.

Also if you bump both buffers to 1 there seems to be a cyclic distortion. Likely some rounding or timer issue. But it happens on other builds too.

@pljones
Copy link
Copy Markdown
Collaborator

pljones commented Apr 29, 2026

It's definitely interesting that this lack of backwards compatibility without

  -F, --fastupdate        use 64 samples frame size mode

on the server when the client is using image hasn't been spotted before -- I've certainly been using Small Network Buffers on every server I connect to and I can't say I've ever noticed anything. Here's my full settings:
image

Is this happening only on rawaudio servers not using -F?

@softins
Copy link
Copy Markdown
Member

softins commented Apr 29, 2026

It's definitely interesting that this lack of backwards compatibility without

  -F, --fastupdate        use 64 samples frame size mode

on the server when the client is using Small Network Buffers hasn't been spotted before -- I've certainly been using Small Network Buffers on every server I connect to and I can't say I've ever noticed anything.
...
Is this happening only on rawaudio servers not using -F?

Yes, it seems so. I've just done some tests with Wireshark capturing the traffic between a client and a server running without -F:

  • Server 3.11.0, client 3.11.0 - no problem.
  • Server latest main, client recent main - no problem.
  • Server latest rawaudio-dev 3f059b3, client recent main - 64 byte buffers with Small Buffers enabled gives the distorted audio, even though the client isn't a rawaudio build.

I have saved the Wireshark captures, but not yet examined them.

@pljones
Copy link
Copy Markdown
Collaborator

pljones commented Apr 29, 2026

I asked VSCode CoPilot:


Likely cause identified: a packet-send regression in server mix/encode on this branch.

The server can require two sub-blocks per client frame when server runs without fastupdate (128-sample server frame) and a client is on small buffers (64-sample transport). That is set here: server.cpp:771.
In this branch, encode/copy still iterates over both sub-blocks, but packet send is done only once after the loop: server.cpp:1191, server.cpp:1205, server.cpp:1213.
Result: when two blocks are needed, only the last encoded/copied block is transmitted each cycle (the first is overwritten before send). That produces the garbled/alternating distortion pattern users report.
Why this matches your symptom:

It is triggered specifically by non-fastupdate server mode (128-frame server path) with 64-frame clients.
That is exactly the compatibility path that depends on two conversion blocks and is sensitive to send placement.


It supplied this patch

@@
-        if ( iCeltNumCodedBytes != static_cast<int> ( iClientFrameSizeSamples * vecNumAudioChannels[iChanCnt] * sizeof ( int16_t ) ) )
+        if ( iCeltNumCodedBytes != static_cast<int> ( iClientFrameSizeSamples * vecNumAudioChannels[iChanCnt] * sizeof ( int16_t ) ) )
         {
             // OPUS encoding
             if ( pCurOpusEncoder != nullptr )
             {
@@
                 for ( int iB = 0; iB < vecNumFrameSizeConvBlocks[iChanCnt]; iB++ )
                 {
                     const int iOffset = iB * SYSTEM_FRAME_SIZE_SAMPLES * vecNumAudioChannels[iChanCnt];
 
                     iUnused = opus_custom_encode ( pCurOpusEncoder,
                                                    &vecsSendData[iOffset],
                                                    iClientFrameSizeSamples,
                                                    &vecvecbyCodedData[iChanCnt][0],
                                                    iCeltNumCodedBytes );
+
+                    // send each encoded sub-block (important when vecNumFrameSizeConvBlocks == 2)
+                    vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );
                 }
             }
         }
         else
         {
             for ( int iB = 0; iB < vecNumFrameSizeConvBlocks[iChanCnt]; iB++ )
             {
                 const int iOffset = iB * SYSTEM_FRAME_SIZE_SAMPLES * vecNumAudioChannels[iChanCnt];
 
                 memcpy ( &vecvecbyCodedData[iChanCnt][0], &vecsSendData[iOffset], iCeltNumCodedBytes );
+
+                // send each raw sub-block (important when vecNumFrameSizeConvBlocks == 2)
+                vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );
             }
         }
-        // send separate mix to current clients
-        vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );

My fault for saying put the PrepAndSendPacket outside the if, I think...

@softins
Copy link
Copy Markdown
Member

softins commented Apr 29, 2026

Wow, that's impressive! I haven't tried copilot yet.

Although I can't see what is different between the two versions of the first line?

I'll try the patch a little later and repeat my tests.

@dingodoppelt
Copy link
Copy Markdown
Contributor Author

I can confirm the issue being fixed with the last commit in which I applied the suggested patch. (actually a revert)

@softins
Copy link
Copy Markdown
Member

softins commented Apr 30, 2026

Yes, I can confirm the problem with small buffers enabled connected to a server without -F appears to be fixed.

Finally understood that the only difference between having -F or not is that with 64-byte packets, when -F is specified, the server sends them one at a time at 1.33ms intervals, and without -F, it sends them in pairs at 2.66ms intervals. The issue before the last commit was that only one of each pair was being sent.

@dingodoppelt
Copy link
Copy Markdown
Contributor Author

Furthermore I can't reproduce the crash on closing anymore. On linux I was getting crashes very rarely in the first place but I could trigger it by connecting to a server with -c <IP> but this doesn't crash anymore when closing the client.
This should be tested on macos and windows, too since I've had reports (mostly from mac users) of the client crashing on close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs documentation PRs requiring documentation changes or additions

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

7 participants